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Abstract 

This study aims to develop a reliable and valid scale to identify the levels of media users' 
media literacy skills. The scale development process was carried out in nine steps as 
recommended in the literature. Before the scale was administered, the items were 
reviewed by field experts and language experts and a pilot study was carried out. Responses 
from 322 pre-service teachers, selected via purposeful sampling, were included in the 
analysis. Item discrimination was tested via item-total correlation and it indicated that none 
of the items were below .30. In the confirmatory factor analysis, it was found out that the 
scale and the theoretical model showed a fit between good and acceptable. Convergent 
validity, divergent validity and 27% upper-lower group means were also examined. As for 
the internal consistency, Cronbach's alpha value of the scale was calculated as .919 and 
alpha values of the factors were calculated as .768, .833, .720 and .838 respectively. The 
results revealed that Media Literacy Skills Scale, which consists of 45 items gathered under 
the four main factors of 'access, analyze, evaluate and communicate', is a reliable and valid 
measurement instrument. This up-to-date scale covers all main skills of media literacy and 
it consists of a sufficient number of questions to obtain rich data and ensure measurement 
precision. In addition, it covers new media as well as mass media and this way it fills a gap 
in the literature caused by the changing nature of media and technology. 

Keywords: Media literacy; Scale development; Media literacy skills; New media, 
confirmatory factor analysis 


Introduction 

In specific periods of time throughout history, the concept of literacy has been defined by 
information and communication technologies (ICTs) commonly used in that particular era, and 
this fact refers to a parallelism between evolutions of literacy and ICTs. As an outgrowth of this, 
literacy has undergone numerous changes over time (Hsu, Ching & Grabowski, 2009). These 
changes can be addressed in universal, locational and timewise dimensions. From the aspect of 
universal dimension, there is a consensus on the contemporary concept of literacy that has been 
put forth by scientific studies. Locational and timewise dimensions are, on the other hand, 
relative. The meaning which is attributed to literacy, literacy skills and qualities of a literate 
individual in a country may vary with the state that country is in with respect to its indicators of 
development (Ivanovic, 2014). Time wisely, the discrepancy between the meanings attached to 
literacy concept in the past and now is the most concrete evidence of this change (Bawden, 
2001 ). 
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The development of ICTs has not only changed the literacy concept on the levels of content and 
application but also has led to the appearance of new types of literacy as an outgrowth of the 
diversification of information and communication sources, channels and applications (Inal, 
2009). Compared to the past, the current century can be regarded as a period in which new 
literacies of different types and qualities are defined in great numbers (Perez Tornero, Celot & 
Varis, 2007). Besides, literacy is also addressed with different concepts regarding the 
fundamental competencies which are defined by associating it with a specific field. Rhetoric, 
speaking and listening, print literacy (Hobbs & Moore, 2013), visual literacy, information literacy, 
media literacy (Bawden, 2001), critical literacy (Lankshear & McLaren, 1993), computer literacy, 
news literacy, digital literacy (Donna Alvermann & Hagood, 2000), gaming literacy, social media 
literacy (Caperton, 2010), internet literacy and multimedia literacy (Hofstetter, 2002) are some 
of them. 

Based on the abovementioned explanations, it can be argued that the twenty first century is a 
period in which ICTs have diversified; media is used so intensely; and communication process 
has transformed into a global action form, independent of restrictive elements (Kellner & Share, 
2007). Particularly in the last century, the internet has become a variable that has played the 
most significant role in the formation of this phase and it has had the potential of directly 
affecting literacy skills in terms of constructing information in various forms, storing and 
communicating and so on. By contributing to the dissemination of information independent of 
time and space, ICTs, particularly computers and the internet, have served a substantial role in 
the emergence of discussions regarding new types of literacy (Onal, 2010). 

As a consequence of the change in the meaning attributed to literacy, the competencies that 
literate individuals need to possess are being redefined. Decoding print texts and producing new 
print texts are now superficial skills for an individual to be regarded as literate. The featured 
competencies of literate people nowadays include critically analyzing messages of different 
types such as texts, images, and audio files, videos in multidirectional and multidimensional 
information and communication channels, and creating texts in different forms. Moreover, in 
this new period, the problem of accessing information has been dissolved thanks to the 
opportunities offered by ICTs. In contrast, avoidance skills to abstain from information sources 
(Bawden & Robinson, 2009) that are constructed, harmful and unreal (Potter, 2008) are 
regarded among the literacy skills. In addition, a literate individual today needs to possess 
competencies listed below (Kellner & Share, 2005; Ayres, Langone & Douglas, 2009; Asici, 2009; 
Onal, 2010): 

• Recognizing and expressing facts, 

• Making use of texts of different types to express one's emotions and thoughts, 

• Reflecting the obtained knowledge to one's own behaviors, 

• Knowing the form of texts shared in different sources and re-creating and using them, 

• Possessing up-to-date knowledge and skills, 

• Communicating effectively, 

• Building one's capacity to be able to provide cultural and social contribution, 

• Expressing oneself through using one's knowledge and skills in social and cultural field. 

Regarding the change in literacy, Kress (2003) pointed out two fundamental factors stemming 
from social, technological and economic variables. The first of these is the move from the 
dominance of writing to the dominance of the image. Consequently, the second one is the 
transition from the dominance of the book to the dominance of the screen as a medium of 
communication. In the past, possessing the skills of decoding, reading and writing was regarded 
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adequate for literacy because literacy was perceived as the skills of phonological awareness or 
resolving information and data appearing in print materials (Lundgren, 2013) since information 
sources were print materials to a considerable extent. Today, on the other hand, the emergence 
of new, different and multidimensional information and communication channels (Freire & 
Macedo, 1998), accession of information to individuals through powerful images and sounds of 
multi-media culture (Thoman & Jolls, 2005) have led to the extension and redefinition of 
competencies of the concept literacy (Silverblatt et al., 2014). The definition of literacy has been 
extended to include digital, electronic and visual expressions (Gentry & McAdams, 2013). 


Media Literacy 

Media literacy is one of the literacy types on which academic studies have been carried out in 
great numbers particularly on technological literacy. There is not a consensus on a single 
definition of media literacy in the literature. Yet, the studies on media literacy are increasing day 
by day (Potter, 2010). Notwithstanding the lack of consensus, media literacy is generally defined 
as 'the ability to access, analyze, evaluate and communicate messages in a variety of forms' 
(Aufderheide, 1993). This widely acclaimed definition includes four main skills- access, analyze, 
evaluate and communicate- as well as using media tools and platforms effectively (Potter, 2009). 
In a media era of rapid technological transformations, a definition of media literacy independent 
of media tools at use is highly functional and significant (Livingstone, 2003). 

Media literacy can be regarded as an umbrella term independent of the variables such as media 
tools at use, technology and so on. Besides, the unprecedented developments and increasing 
variety in ICTs, particularly the potentials offered by Web 2.0, have had a significant role in the 
emergence of new forms of literacy such as new media literacy and social media literacy (Walsh, 
2010). Furthermore, when compared to the past, the meaning attributed to media today has 
changed and expanded to some extent since media today is quite different from the media in 
the past. 

According to Rivoltella (2006 as cited in Perez Tornero, Celot & Varis, 2007), today's media is 
different from the past in five ways. The new media differs from the old media as it embodies 
multimediality of instruments and contents, portability, high connectivity, multimediality of 
consumption, and high interactivity. Andersen (2002) highlights the advantages of the new 
media in terms of hardware and software. In addition to these advantages, he asserts that media 
literacy has to do with the thinking processes that are needed to use media contents and create 
new ones. He maintains that the new media embodies a multidirectional communication and 
synchronicity while the old media enables mainly one directional communication. Beside 
interactivity and simultaneity, synchronicity here includes miniaturization of media tools, 
portability, speed, access, and convergence of all these properties in a single medium. 

Chen, Wu and Wang (2011) argue that the digital representation of the new media makes it 
programmable and computable. Besides, the new media is modular; namely, single small 
modules can be combined into new bigger modules to achieve different goals. They contend 
that these two characteristics of the new media enable the automation of operations and 
diversity in media creation and use. Regarding abovementioned explanations, the prominent 
features of new media are automation, portability, diversity, digitality, instant dissemination, 
modularity, multi-layeredness, hybridity and interaction (Andersen, 2002; Livingstone, 2004; 
Rivoltella, 2006; Jenkins et al., 2006; Anderson & Balsomo, 2008; Chen et al., 2011). 
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Since the media is not only a factor affecting the emergence of the new participatory culture but 
also has become a part of that culture (Jenkins et al., 2006), today's media can only be 
understood when socio-cultural characteristics of it are also taken into account. Socio-cultural 
characteristics of the media can be assessed in three aspects: (a) media messages are 
constructed and fictionalized and therefore reflect only one side of the reality; (b) ideological 
and social values are integrated into the media; and (c) the media serves for political and 
economic purposes among others. Besides, another socio-cultural characteristic of the new 
media is that each medium has its own peculiar language (Chen et al., 2013). 

From the socio-cultural aspect, it can be argued that today's media offers alternative 
opportunities for people. The experiences in the digital revolution and the process of formation 
of information society reveal that people tend to adopt new technologies they encounter 
without questioning (Perez Tornero & Varis, 2010). This brings on an important problem. In mass 
communication, media conglomerates try to protect and expand their power and dominance by 
filtering information and manufacturing consent as well as controlling what people watch, listen 
to, think, believe, feel and perceive (Torres & Mercado, 2010). However, the new media comes 
forth as an alternative to media conglomerates and makes it possible for individuals to share 
with other people what they have created via new media tools. 

The progress brought by media tools and applications requires media literacy to be re-evaluated 
within the context of the twenty first century (Chen et al., 2011). Media literacy studies and 
research are to focus on new media tools, platforms and messages such as the internet and 
mobile phones besides traditional media tools (Literat, 2014). Media literacy field should focus 
not only on media consumption but also on the new media and the participatory culture that 
develops along with the new media (Jenkins et al., 2006). Therefore, the focus of media literacy 
has expanded to include the internet and other new media in addition to traditional print and 
audio-visual media (Livingstone, 2003). 

Digital technology has placed media into the center of people's lives who now live in a digital 
balloon (Perez Tornero & Varris, 2010). The development of new communication technologies 
has transformed media consumers into media creators and the creation, sharing, and 
dissemination of original media messages are now defined among the characteristics of qualified 
media users (Literat, 2014). As people today face a bombardment of visual, audial and 
multimedia messages in every walk of life from televisions, newspapers and billboards to radios, 
sales catalogues and the internet, media literacy skills are of vital importance (inal, 2009). 
Besides, schools now use media tools intensely for instruction and students' extensive use of 
media and educational technology requires them to possess media literacy skills. Therefore, 
media literacy skills are not a matter of preference but a necessity for individuals. 


Media Literacy Skills 

The skills and competencies that a media literate individual needs to possess are among the 
much-discussed issues in the literature. European Charter for Media Literacy, for instance, lists 
seven key competencies that media literate people should have and these include using media 
effectively, accessing and making informed choices about media content, understanding media 
content creation, analyzing media techniques and messages, using media to communicate, 
avoiding harmful media content and services, and using media for democratic rights and civic 
purposes (Bachmair & Bazalgette, 2007). In addition to these competencies, literature highlights 
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four main skills to be an effective media literate. These skills include access, analyze, evaluate 
and communicate (Aufderheide, 1993; Hobbs, 2001; Livingstone, 2003; Jolls, 2008; Silver, 2009; 
Schmidt, 2013; Silverblatt at al., 2014; Sahin, 2014). The explanations and contents of these skills 
are provided below. 

Access involves locating and using appropriate media tools (Hobbs, 2010), reaching the targeted 
information via these tools, and comprehending the meanings of the contents (Jolls, 2008). For 
the access skill, media tools like mobile phones, ipads, televisions and computers and, more 
importantly, technological knowledge of a certain level are needed (Bilici, 2014). For an 
individual to participate in the media culture as a creator or consumer, it is prerequisite to 
possess the competencies within the scope of access (Schmidt, 2013). However, it should be 
noted that physical access to media messages or media tools does not guarantee effective use 
of them. With respect to the access skill which has social, cultural and technological aspects, 
factors such as having the permission to use media tools, knowing the use of software, peer- 
group norms regarding accepted actions are important (Livingstone, 2003). However, access is 
not limited to one-time purchase or supply as it accommodates a dynamic and social process 
(Livingstone, 2004). Therefore, what is important is the sustainability of use. In this sense, access 
can be regarded as consisting of two phases, which are physical access to media and contents 
of media, and the ability to use the media properly (Perez Tornero, Celot & Varis, 2007). Hobbs 
& Moore (2013) list competencies of access skills as listening skills, comprehending content, 
using appropriate technology tools, asking questions, gathering information using multiple 
sources, implementing information to solve a problem. 

Analyze may be regarded as the most prominent skill of media literacy. This is because the main 
goal of media literacy is to enable critical autonomy in the relation with media (Aufderheide, 
1993), and the ability to analyze symbolic texts underlies media literacy (Livingstone, 2003). 
After accessing media contents, a set of competencies is needed to analyze those. These 
competencies include one's examining the structure, content, design, form and sequence of the 
messages with a critical perspective using artistic, literary, social, political and economic 
concepts (Jolls, 2008). 

Analytical skill refers to dividing messages into meaningful units (Kellner, 2001). When 
individuals encounter media messages, they either adopt these messages superficially or they 
break these messages into parts and get deep into the messages by examining these parts 
separately (Potter, 2008). In this process, authorship, audience, form and type of the message 
and points of view, characters, subjects and themes, mood, setting and context in these 
messages are analyzed (Thoman & Jolls, 2005; Hobbs & Moore, 2013; Bachmair & Bazalgette, 
2007). A media literate individual with analytical skills is aware that a media message is a product 
of planned work and clearly sees the limit between the real world and the world virtually created 
by the media (Potter, 2008). Besides, a media literate individual analyzes literary strategies, 
possibilities and context (Lewis & Jhally, 1998); knows how media symbols are used (Baran, 
2014); knows that media uses specific techniques to create emotional influence (Bachmair & 
Bazalgette, 2007); and perceives the implicit meaning as well as the explicit meaning in the 
messages (Volvic, 2003). Finally, a media literate individual needs to have a sufficient grasp of 
the background, structure and functioning of media industry to be able to comprehend and 
discuss what is offered through media (Taskiran, 2007). 

Evaluate is a process to complete the analysis phase and a skill to make judgements about the 
quality or the value of a media content. Without evaluation skills, phases of access and analyze 
would remain inconclusive. Making a judgement is performed via comparing media content with 
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a standard or a value measure (Perez Tornero, Celot & Varis, 2007). These values may be ethical, 
moral, scientific or democratic principles (Jolls, 2008). In the evaluation process, taking the 
potential effects or outcomes of the messages into account is also important (Hobbs, 2010). 
When encountered with opinions expressed in media messages, individuals either memorize 
and adopt them as their own opinions or compare the information elements in the message 
with their own standards (Potter, 2008). In this phase, one needs to evaluate the media content 
that they accessed and analyzed previously. In the evaluation process, one relates the messages 
to one's own experiences and makes judgements about the reality and quality of the messages 
(Thoman & Jolls, 2005). In addition, when interpreting messages of different types and forms, a 
media literate individual identifies and avoids unsolicited and harmful media content (Bachmair 
& Bazalgette, 2007; Silver, 2009), and evaluates the objectivity of the information in media 
messages with advertisement content (Sahin, 2014). 

Majority of media messages are constructed with commercial, political or other purposes. 
Therefore, the evaluation process in which individuals question media contents with respect to 
objectivity and reality is of vital importance for media literacy. In the new media order, in which 
anyone can create one's own contents and share them with other individuals, and in which there 
are not sharp borders between media creators and consumers, questioning the quality of 
content is quite different from print texts. That is why individuals need to possess broad 
knowledge structures regarding social, cultural, economic, political and historic contexts to 
make a critical evaluation and perceive the internet not as a source of true information but as a 
resource whereby information is questioned, evaluated and reflected upon (Livingstone, 2003). 

Communicate comprises of competencies regarding creating media messages and sharing them 
with other people (Schmidt, 2013). The skill of communicate is quite significant in the sense that 
it converts media consumers into media creators and distributors. Through this skill which 
enables media users to create and share their own media messages as an alternative to 
professionally created media contents, individuals learn the processes of content selection, 
editing and construction methods, techniques and technologies by doing and by this means 
comprehend the media kitchen way better (Bilici, 2014). 

In addition to these four skills, there is a fifth skill recently discussed in the literature. This skill is 
called "participation" (Jolls, 2008, Thoman & Jolls, 2005) or "act" (Hobbs, 2010) and it involves 
competencies regarding being an active citizen via using media tools to find a resolution to 
problems in family, business, social circle, or society. This skill is quite related to the 
'communicate' skill and is actually a part of it. Whereas only certain information and certain 
points of view are available in mainstream media, new media platforms have enabled those who 
cannot make their voice heard to express themselves particularly with technologies that allow 
instant interaction. Through creating content and communicating them, individuals can actively 
participate in social life, support various campaigns and become active citizens who can defend 
their rights. In this study, the skill of communicate is regarded as a meta-concept involving the 
skill of participate since it involves content creation and sharing this with other people and thus 
participating in social life. 

Measuring levels of individuals' media literacy skills is highly significant to identify users' 
weaknesses and provide them with relevant training. To this end, various scales have been 
developed (Chang et al., 2011, Ashley Maksl & Craft, 2013; Literat, 2014). In the Turkish context, 
two scales, developed by Karaman & Karatas (2009) and Korkmaz & Yesil (2011), are used 
predominantly in the studies on media literacy (Som & Kurt, 2012; Engin & Gene, 2016; Aybek 
& Demir, 2014; Cepni, Palaz & Ablak, 2015). However, scales on media literacy need to be 
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updated periodically to cope with the fast-changing nature of media and technology. In the last 
ten years, the development of new media tools and platforms has been pervasive and has 
featured particularly the "communicate" skill. A literature review on existing scales reveals that 
an up-to-date media literacy skills scale with enough questions to ensure rich data and 
measurement precision to cover all sub-skills of media literacy and to involve new media besides 
mass media is needed and this study aims to fill this gap. 


Purpose of the Study 

The main purpose of this study is to develop an up-to-date, reliable, and valid media literacy 
skills scale based on the media literacy skills in the literature. The reason for choosing media 
literacy concept in this study is to define the skills that fit to all media tools and basic skills 
independent of the tools and applications used as opposed to skills particular to certain media 
tools or platforms. It is acknowledged that defining media literacy skills regardless of media tools 
used is highly functional and significant in an era of rapid technological transformation. To this 
end, research questions in this study are as follows: 

1. Is media literacy skills scale a valid measurement tool that can be used to determine the 
level of individuals' media literacy skills? 

2. Is media literacy skills scale a reliable measurement tool that can be used to determine 
the level of individuals' media literacy skills? 


Methodology 

In this study, scale development process was carried out in nine steps as offered in the literature 
(DeVellis, 2003; Tavsancil, 2006; Tekindal, 2009). These steps included; (1) literature review to 
clearly define what is aimed to be measured and its components, (2) generating an item pool, 
(3) determining the appropriate format for measurement; (4) development of the draft 
measurement tool, (5) review of items by experts, (6) pilot study, (7) administration of the 
measurement tool to a development sample, (8) carrying out reliability and validity studies, (9) 
finalizing the measurement tool. In the following, studies done in each step are explained in 
detail respectively. 


Scale Development Process 

In the first phase, the literature was reviewed to define media literacy clearly and to generate 
an item pool. Based on the literature review, the characteristics that a media literate individual 
needs to possess were identified. After an extensive literature review (Aufderheide, 1993; 
Bachmair & Bazalgette, 2007; Baran, 2014; Bilici, 2014; EAVI, 2011; Jenkins et al., 2009; Hobbs 
& Moore, 2013; Kellner, 2001; Lewis & Jhally, 1998; Volvic, 2003; Koltay, 2011; Literat, 2014; 
Potter, 2008; Silver, 2009; Silverblatt et al., 2014; Taskiran, 2007; Thoman & Jolls, 2005), a list of 
166 items was formed. These items were analyzed and overlapping items were combined, which 
reduced the item number to 113. The items on the reduced list were grouped under four basic 
media literacy skills; access, analyze, evaluate and communicate as offered in the literature. 
Once grouped, the items were re-examined in terms of meaning, measuring the same 
characteristics etc. based on the dimension-wise classification and the item number was 
reduced to 82. 
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After item selection, the appropriate format for measurement was selected. For the 
measurement of levels of media literacy skills, Likert type was chosen due to its ease, high 
reliability and validity as well as its successful use in measuring affective variables (Tekindal, 
2009). Participants chose among the options of "(1) Completely unsuitable for me" "(2) Not 
really suitable for me" "(3) No idea" "(4) Quite suitable for me" and "(5) Completely suitable for 
me" based on their competencies on a given item. 

For the content validity of the scale and the appropriateness of the items for measuring media 
literacy skills, field experts were consulted. The experts were professors working in education 
and communication faculties of various universities in Turkey. The researchers created an 
assessment form for experts to score the items and add feedback, and they e-mailed it to 
experts. Depending on the feedback from 6 experts, 22 items were omitted from the draft scale 
and item number was reduced to 60. Next, the draft scale was sent to five language experts 
who were Turkish language teachers with experience of at least five years in Turkey and 
necessary changes were made on the scale regarding spelling, punctuation and language use 
based on expert feedback. The pilot study consisted of two phases. In the first phase, 17 
freshmen students at a state university took the draft scale. In the second phase, a discussion 
was held with students regarding the scale and the expression level of the items. The discussion 
was on how the students perceived the items, the items that they had difficulty in 
understanding, and the face validity of the draft. Based on the feedback, 13 more items were 
omitted and a scale consisting of 47 items were administered to development sample. 


Participants 

Purposeful sampling (Creswell, 2002) was adopted for participant selection. The data of the 
study were gathered from 322 university students. The criteria considered in determining the 
sample size were that sample size should be at least five times the number of items (Sencan, 
2005; Tavsancil, 2006) and that a sample size of at least 300 participants is needed for factor 
analysis (Field, 2009). The participants were students studying at the education faculty of a state 
university in Turkey in 2015-2016 academic year. Participants' gender, department and grade 
information is given in Table 1. 


Table 1. Demographics of the Participants 


Department 

Male 

Female 

Total 

Comp. Ed. Tech. Ed. 

Grade 1 

15 

14 

29 

Science Ed. 

Grade 1 

12 

39 

51 

Grade 2 

7 

28 

35 

Primary Math. Ed. 

Grade 1 

5 

24 

29 

Grade 2 

5 

30 

35 

Preschool Ed. 

Grade 1 

6 

31 

37 

Grade 2 

1 

32 

33 

Turkish Ed. 

Grade 1 

20 

20 

40 

Grade 2 

9 

24 

33 

Total 

80 

242 

322 
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Procedures 

The data collection procedure was carried out in May, 2016. Permission was obtained before 
data collection and the students were informed about the study and forms were given to only 
volunteering students. The total number of students in the randomly selected departments and 
grades was 612. 394 students took the form and 72 students' responses were not included in 
the analysis as they were not completed appropriately and the analysis was performed with 322 
responses. 


Findings 

This part includes analysis results of the construct validity and reliability. For the construct 
validity, confirmatory factor analysis (CFA) was performed. CFA is sounder than exploratory 
factor analysis because it provides more reliable data theoretically about the model and factor 
structure (Sencan, 2005). CFA is used when a model is constructed based on theory or previous 
studies and when the fit level between the model and data set is tested (Joreskog & Sorborn, 
1993). Therefore, the researcher needs to know the factor structure of the model and which 
variables belong to which factor and whether factors are related to each other (Thompson, 
2004). The scale developed in this study is based on the hypothesis that media literacy comprises 
four factors grounded on the information in the literature. Therefore, CFA was considered as 
the best method for testing construct validity. Besides, convergent validity and discriminant 
validity and means of 27% upper-lower groups were also checked. For reliability, Cronbach a 
value was calculated. 


Item Discrimination 

Item discrimination is used to test the extent to which the items measure the construct 
(Buyukozturk, 2010). Before factor analysis, item discrimination of the items in the scale was 
tested. To this end, item-total correlation values were calculated and the observed values are 
given in Table 4. Item-total correlation value of an item should not be below .30 (Field, 2009). In 
the analysis, none of the items were below .30 and so items were appropriate for factor analysis. 


Mean Differences between 27% Upper-Lower Groups 

The differences between mean scores of upper 27% and lower 27% were examined. To this end, 
the sum of the scores were listed ascending and groups of upper 27% and lower 27% were 
formed and independent sample t-test was carried out. The t-test results for 27% upper-lower 
group means are given in Table 2. The t-test results showed significant differences between the 
means of the two groups [t(322)=28.401, p<.05]. 

Table 2. t-test results for 27% upper-lower group means 


Groups 

N 

X 

SD 

df 

t 

P 


Upper 27% 

87 

138.91 

12.37 

172 

28.401 

.000 


Lower 27% 

87 

187.09 

9.86 

163.84 

28.401 

.000 
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Construct Validity 

Before CFA, Kaiser-Meyer-Olkin Measure of Sample Adequacy (KMO) was checked to test 
whether the sample size is adequate for factor analysis. KMO values vary between 0 and 1 and 
it is observed as .876 in this study which is considered as "very good" in the literature (Tavsancil, 
2006). This value reveals that sample size is adequate for further analysis. CFA was carried out 
to test whether the theoretical model offered in the scale fits the data set. The analysis program 
(Lisrel 8.51) suggested a modification that there would be a significant decrease in chi-square 
value if two of the items were omitted. Therefore, items 9 and 24 were omitted as suggested by 
the program. First, t values of all items were above 2.56 and therefore they were significant 
(p<.01) and are given in Table 4. Next, the fit measures were checked. Table 3 provides 
acceptable fit indices (Schermelleh-Engel, Moosbrugger, & Muller, 2003) and the related values 
observed in this study. 

Table 3. Acceptable Values for Fit Measures and Observed Values 


Fit Measure 

Acceptable values 

Observed Values 

x2/sd 

0 < x2/df < 2 good fit 

2 < x2/df < 3 acceptable fit 

2.00 

GFI 

.95 < GFI < 1.00 good fit 
.90 < GFI < .95 acceptable fit 

.95 

AGFI 

.90 < AGFI < 1.00 good fit 
.85 < AGFI <.90 acceptable fit 

.94 

NFI 

.95 < NFI <1.00 good fit 
.90 < NFI < .95 acceptable fit 

.94 

NNFI 

.97 < NFI < 1.00 good fit 
.95 < NFI < .97 acceptable fit 

1.00 

CFI 

.97 < CFI < 1.00 good fit 
.95 < CFI < .97 acceptable fit 

1.00 

RMSEA 

0 < RMSEA < .05 good fit 

.05 < RMSEA < .08 acceptable fit 

0.056 

SRMR 

0 < SRMR < .05 good fit 

.05 < SRMR < .08 acceptable fit 

0.059 

PGFI 

0 < PGFI < 1 

more parsimonious closer to 1 

0.86 


GFI= Goodness-of-Fit Index AGFI= Adjusted Goodness-of-Fit Index 

NFI= Normed Fit Index NNFI= Nonnormed Fit Index 

CFI= Comparative Fit Index RMSEA= Root Mean Square Error of Approximation 

SRMR= Standardized Root Mean Square Residual PGFI= Parsimonious Goodness of Fit Index 


As seen in Table 3, fit indices of the scale showed a fit between good and acceptable. First, the 
ratio of degree of freedom (939) to chi-square value (1882.68) indicated a good fit (x2/df=2.004). 
In addition to the chi-square goodness of fit, other fit indices were examined and found to be 
good or acceptable. Goodness-of-Fit Index (GFI: .95), Adjusted Goodness-of-Fit Index (AGFI: .94), 
Nonnormed Fit Index (NNFI: 1.00) and Comparative Fit Index (CFI: 1.00) indicated good fit. 
Normed Fit Index (NFI: .94), Root Mean Square Error of Approximation (RMSEA: .056) and 
Standardized Root Mean Square Residual (SRMR: .059) indicated acceptable fit. Besides, 
Parsimonious Goodness of Fit Index (PGFI: .86) indicated a parsimonious fit. These values reveal 
that the model shows good and acceptable fit with the data set. For convergent validity, factor 
loads and construct reliability were examined. Factor loads of the items are provided in Table 4 
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and they are moderate and statistically significant, providing evidence for convergence validity. 
For construct reliability, square of factor loads sum was divided by the sum of square of factor 
load and error variances sum. This value needs to be over .60. The result was .94, which refers 
to convergent validity. For divergent validity, correlations between factors should not be 
excessively high (e.g. <.90) (Kline, 2011). Correlations between factors are provided in Table 5 
and the values are not high, which indicates divergent validity. Item statistics are provided in 
Table 4 and the diagram of confirmatory factor analysis indicating the distribution of items under 
each factor is provided in Figure 1. English translations of 8 sample items are provided in the 
Appendix. 

Table 4. Item-Total Correlations and Item Statistics of Confirmatory Factor Analysis 


Item 

r 

t value 

Error 

Variance 

Factor 

Load 

X 

SD 

Iteml 

.511** 

20.18 

0.67 

0.57 

3.87 

0.78 

Item2 

44^** 

22.54 

0.75 

0.50 

3.72 

1.03 

Item3 

.433** 

19.22 

0.78 

0.46 

3.55 

0.91 

Item4 

455 ** 

18.39 

0.76 

0.49 

3.90 

0.82 

Item5 

.394** 

17.29 

0.82 

0.42 

3.73 

0.90 

Item 6 

.458** 

17.87 

0.75 

0.50 

3.73 

0.79 

Item7 

.506** 

22.94 

0.68 

0.56 

3.32 

0.92 

Item 8 

.428** 

20.46 

0.81 

0.43 

2.88 

1.05 

Item9 

.404** 

18.36 

0.83 

0.41 

3.23 

0.99 

ItemlO 

423 ** 

18.69 

0.80 

0.45 

3.82 

0.91 

Itemll 

.529** 

24.07 

0.67 

0.57 

3.75 

0.96 

Iteml2 

.384** 

19.34 

0.84 

0.40 

3.56 

1.04 

Iteml3 

.462** 

21.68 

0.73 

0.52 

3.63 

0.92 

Iteml4 

452** 

22.26 

0.76 

0.49 

3.33 

0.99 

Iteml5 

.488** 

22.02 

0.72 

0.53 

3.63 

0.90 

Iteml 6 

.484** 

22.70 

0.73 

0.52 

3.33 

0.94 

Iteml7 

.418** 

18.02 

0.81 

0.44 

3.55 

0.87 

Iteml 8 

.555** 

24.10 

0.66 

0.58 

3.19 

0.89 

Iteml9 

.500** 

21.55 

0.68 

0.57 

3.88 

0.82 

Item20 

454 ** 

21.98 

0.75 

0.50 

3.50 

0.95 

Item21 

.481** 

24.59 

0.73 

0.52 

3.47 

1.02 

Item22 

472** 

19.83 

0.74 

0.51 

3.85 

0.83 

Item23 

.481** 

22.17 

0.73 

0.52 

3.68 

0.93 

Item24 

.481** 

22.05 

0.70 

0.55 

3.98 

0.87 

Item25 

.423** 

19.18 

0.77 

0.48 

3.87 

0.86 

Item26 

.483** 

23.36 

0.72 

0.53 

3.57 

0.95 

Item27 

.407** 

18.07 

0.75 

0.50 

3.81 

0.92 

Item28 

.489** 

20.52 

0.64 

0.60 

3.75 

0.90 

Item29 

. 449 ** 

21.96 

0.69 

0.56 

3.53 

1.05 

Item30 

.387** 

16.96 

0.78 

0.47 

4.14 

0.89 

Item31 

.435** 

18.24 

0.73 

0.52 

3.66 

0.88 

Item32 

.430** 

19.20 

0.71 

0.54 

3.81 

0.89 

Item33 

425 ** 

19.86 

0.75 

0.50 

3.58 

1.00 

Item34 

4g2** 

25.07 

0.70 

0.54 

3.84 

1.00 
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Item35 

.552** 

26.80 

0.64 

0.60 

3.67 

0.98 

Item36 

47?** 

20.68 

0.75 

0.50 

3.62 

0.88 

Item37 

.516** 

25.22 

0.65 

0.59 

3.89 

0.92 

Item38 

.533** 

26.86 

0.66 

0.58 

3.61 

1.00 

Item39 

. 434 ** 

23.09 

0.77 

0.48 

3.54 

1.03 

Item40 

.514** 

28.66 

0.68 

0.57 

3.47 

1.11 

Item41 

.531** 

25.75 

0.65 

0.59 

3.74 

0.94 

Item42 

.488** 

23.81 

0.69 

0.56 

3.72 

0.92 

Item43 

.394** 

20.44 

0.83 

0.42 

2.93 

1.04 

Item44 

.515** 

27.22 

0.68 

0.57 

3.83 

1.04 

Item45 

.577 

30.61 

0.58 

0.65 

3.49 

1.04 


**p<.01, n=322 




Figure 1. Confirmatory Factor Analysis Diagram of Media Literacy Skills Scale 
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Reliability 

Reliability of the scale was tested via the internal consistency test. Internal consistency is that 
items in the scale are measuring the same construct in relation to each other and reliability is 
mostly identified by Cronbach alpha value (Field, 2009). Internal consistency calculations 
revealed that Cronbach Alpha value of the scale is (a=.919). The alpha values for the total of the 
scale and for the factors are given in Table 5. 

Table 5. Reliability Test Results and Correlations between Factors 


Factors 

Alpha Value 

Correlations btw. Factors 

12 3 4 

Access 

.768 

1.0 

.57 

.43 

.62 

Analyze 

.833 


1.0 

.54 

.52 

Evaluate 

.720 



1.0 

.46 

Communicate 

.838 




1.0 

Total 

.919 






According to the criteria regarding the appropriateness of alpha coefficient, values between .70 
and .90 refer to a high level of reliability and values over .90 refer to very high level of reliability 
(Ozdamar, 2011). Field (2009) accepts alpha values of .7 and .8 as a reliable measure. These 
criteria indicate that the scale developed in this study is an instrument with a very high reliability. 


Results and Conclusion 

This study proposes an up-to-date media literacy skills scale. The scale was tested in terms of 
validity and reliability. Scale development process followed nine steps as explained in detail in 
the methodology section. After the literature review, an item pool was generated and items 
were grouped under four factors, access, analyze, evaluate and communicate. After the 
examination of items in terms of overlap, meaning, and measuring the same characteristics, a 
list of 82 characteristics was formed. 22 items were omitted after the expert opinion phase 
carried out for content validity and 13 items were later omitted after the pilot study. Eventually, 
a scale form of 47 items was administered to development sample selected through purposeful 
sampling. Responses from 322 participants were included in the validity and reliability analyses. 
First step in the analysis process was to examine item discrimination indices of the items and it 
indicated that item-total correlation values of all items were above .30. Mean differences 
between 27% upper-lower groups were found statistically significant. CFA was carried out to 
reveal the construct validity of the scale. The analysis revealed that the model showed good and 
acceptable fit with the data set after omitting two items from the scale. 

Media literacy skills are classified differently by various researchers. Potter (2008) classifies 
media literacy skills as analysis, evaluation, grouping, induction, deduction, synthesis and 
abstraction. Hobbs & Moore (2013) suggests that media literacy skills are access, analysis, 
construction, reflection and act. These skills or factors can be diversified; however, the widely- 
accepted definition includes skills of access, analyze, evaluate and communicate. These basic 
skills are highly comprehensive of other skills and fit to all media tools, which makes them 
functional. The proposed scale consists of these factors. In the scale, access includes 11 items, 
analyze includes 15 items, evaluate includes 7 items and communicate includes 12 items. In 
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addition, the scale was tested for convergent and divergent validity. Reliability analysis revealed 
that internal consistency coefficient of the scale is (a=.919), an indication of very high level of 
reliability. 

Although there are various scales in the literature (Chang et al., 2011, Ashley Maksl & Craft, 
2013; Literat, 2014), there are two scales in Turkish used in media literacy studies. The former is 
"Media Literacy Level Determination Scale" developed by Karaman & Karatas (2009) and the 
latter is "Scale of Media and Television Literacy Levels" developed by Korkmaz & Yesil (2011). 
"Media Literacy Level Determination Scale" developed by Karaman & Karatas in 2009 is a Likert 
type scale with 17 items gathered under 3 factors. The factors include "being knowledgeable" 
(7 items), "analyzing and reacting" (6 items) and "judging, being aware of the implicit messages" 
(4 items). "Scale of Media and Television Literacy Levels" developed by Korkmaz & Yesil in 2011 
is a Likert type scale with 18 items gathered under two factors. The factors are "literacy" (13 
items) and "addiction" (5 items). 

The fast-changing nature of media and technology brings about some limitations to the existing 
scales since the media tools people use in daily life or the ones used by students in schools 
change rapidly. For instance, the former scale was developed nearly ten years ago when smart 
phones were not as common. Another issue with the existing scales is that they do not yield rich 
information with respect to sub-skills of media literacy. First, the number of questions are not 
sufficient to provide detailed data regarding levels of media literacy sub-skills. Rather than 17 
questions, 45 questions would provide more insight about their levels of media literacy skills and 
more measurement precision as well. Second, sub-skills of media literacy are not fully reflected 
in the factors of these scales. Particularly the sub-skill of "communicate" is not represented. 
Third, the questions in these scales are mostly about mass communication tools. Particularly the 
latter scale mostly consists of questions on television. However, in this new era, questions on 
new media should be predominant as people are now living in a digital world and they are fully 
engaged in new media. Thus, this scale aimed at meeting these limitations by comprehending 
all main skills of literacy; asking 45 questions to obtain rich data and ensure measurement 
precision and covering new media besides mass media and filling a gap in the literature. 

In this study, a scale with 45 items under 4 factors aiming to determine the levels of media users' 
media literacy skills has been developed. Validity and reliability analyses revealed that the scale 
is a valid and reliable data collection instrument. This scale is significant because (a) the factors 
of the scale are based on the basic skills highlighted in the literature; (b) there is a sufficient 
number of questions to obtain rich data regarding the levels of individuals' media literacy skills; 
(c) analysis results indicate a good fit; and (d) the items of the scale mostly address new media 
tools and platforms besides mass media. With this scale, levels of media users' media literacy 
skills and their strengths and weaknesses can be identified. These data can be used as indicators 
of a potential training and as a needs assessment study for developing curriculum on media 
literacy. Particularly, determining students' levels of media literacy skills helps instructors 
identify the issues they need to focus while using media tools as part of educational technology. 
Additionally, determining prospective teachers' levels of media literacy skills and training them 
based on these data would be helpful as they will be teaching students in the new technology 
environment. This scale was validated with prospective teachers at a state university. Further 
research may address participants of different ages and backgrounds such as high school 
students, university students at different departments, teachers or other individuals and validity 
and reliability values and factor structure of the scale can be tested. 
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Appendix: Sample Items 


Factor: Access 

I can effectively use various media tools to access the information, audio, image or other data 
that I need. 

In order to access the information or data I need, I can functionally use different search engines 
and databases. 

Factor: Analyze 

I question the media contents that I examine with respect to the purposes they were 
constructed and shared for. 

I question whether there is an implicit meaning and purpose or not in the messages 
communicated via media tools besides the explicit meaning and purposes that are perceived 
immediately by readers. 

Factor: Evaluate 

I can identify vicious and harmful media contents and I can protect myself from this kind of 
contents. 

I can evaluate media contents or messages in media in terms of ethical principles and make a 
judgement about them. 

Factor: Communicate 

I can evaluate the possible consequences in case I share messages that contain unreal and 
purposeful information and images and that belong to other people and require permission to 
share. 

Using media tools and platforms, I can arrange activities such as digital campaigns, discussion 
forums and blogs in order to find a solution to various problems or generate social sensitivity for 
a particular issue. 
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